1 . Decision Theory and Testing Simple Hypotheses

1.1 Deciding between two simple hypotheses: the Neyman-Pearson Lemma. Probability theory is reviewed in Appendix D. Suppose an experiment has a set X of possible outcomes. The outcome has some probability distribution µ defined on X. In statistics, we typically don't know what µ is, but we have hypotheses about what it may be. After making observations we'll try to make a decision between or among the hypotheses. In general there could be infinitely many possibilities for µ, but to begin with we're going to look at the case where there are just two possibilities, µ = P or µ = Q, and we need to decide which it is. For example, a point x in X could give the outcome of a test for a certain disease, where P is the distribution of x for those who don't have the disease and Q is the distribution for those who do. Often, we have n observations independent with distribution µ. Then X can be replaced by the set X n of all ordered n-tuples (x 1 ,. .. , x n) of points of X, and µ by the Cartesian product measure µ × · · · × µ of n copies of µ. In this way, the case of n observations x 1 ,. .. , x n reduces to that of one " observation " (x 1 ,. .. , x n). The probability measures P and Q are each defined on some σ-algebra B of subsets of X, such as the Borel sets in case X is the real line R or a Euclidean space. A test of the hypothesis that µ = P will be given by a measurable set A, in other words a set A in B. If we observe x in A, then we will reject the hypothesis that µ = P in favor of the alternative hypothesis that µ = Q. Then P (A) is called the size of the test A (at P). The size is the probability that we'll make the error of rejecting P when it's true, i.e. when µ = P , sometimes called a Type I error. On the other hand, Q(A) is called the power of the test A against the alternative Q. The power is the probability that when Q is true, the test correctly rejects P and prefers Q. The complementary probability 1 − …